Beyond A/B Testing: Sequential Randomization for Developing Interventions in Scaled Digital Learning Environments

Randomized experiments ensure robust causal inference that are critical to effective learning analytics research and practice. However, traditional randomized experiments, like A/B tests, are limiting in large scale digital learning environments. While traditional experiments can accurately compare two treatment options, they are less able to inform how to adapt interventions to continually meet learners' diverse needs. In this work, we introduce a trial design for developing adaptive interventions in scaled digital learning environments -- the sequential randomized trial (SRT). With the goal of improving learner experience and developing interventions that benefit all learners at all times, SRTs inform how to sequence, time, and personalize interventions. In this paper, we provide an overview of SRTs, and we illustrate the advantages they hold compared to traditional experiments. We describe a novel SRT run in a large scale data science MOOC. The trial results contextualize how learner engagement can be addressed through inclusive culturally targeted reminder emails. We also provide practical advice for researchers who aim to run their own SRTs to develop adaptive interventions in scaled digital learning environments

Integrating knowledge tracing and item response theory: A tale of two frameworks

Traditionally, the assessment and learning science commu-nities rely on different paradigms to model student performance. The assessment community uses Item Response Theory which allows modeling different student abilities and problem difficulties, while the learning science community uses Knowledge Tracing, which captures skill acquisition. These two paradigms are complementary - IRT cannot be used to model student learning, while Knowledge Tracing assumes all students and problems are the same. Recently, two highly related models based on a principled synthesis of IRT and Knowledge Tracing were introduced. However, these two models were evaluated on different data sets, using different evaluation metrics and with different ways of splitting the data into training and testing sets. In this paper we reconcile the models' results by presenting a unified view of the two models, and by evaluating the models under a common evaluation metric. We find that both models are equivalent and only differ in their training procedure. Our results show that the combined IRT and Knowledge Tracing models offer the best of assessment and learning sciences - high prediction accuracy like the IRT model, and the ability to model student learning like Knowledge Tracing

A922 Sequential measurement of 1 hour creatinine clearance (1-CRCL) in critically ill patients at risk of acute kidney injury (AKI)

Meeting abstrac

Using data from real and simulated learners to evaluate adaptive tutoring systems

Classification evaluation metrics are often used to evaluate adaptive tutoring systems-programs that teach and adapt to humans. Unfortunately, evidence suggests that existing convention for evaluating tutoring systems may lead to suboptimal decisions. In a companion paper, we propose Teal, a new framework to evaluate adaptive tutoring. In this paper we propose an alternative formulation of Teal using simulated learners. The main contribution of this novel formulation is that it enables approximate inference of Teal, which may useful on the cases that Teal becomes computationally intractable. We believe that this alternative formulation is simpler, and we hope it helps as a bridge between the student modeling and simulated learners community

Challenges of using observational data to determine the importance of example usage

Educational interventions are often evaluated with randomized control trials, which can be very expensive to conduct. One of the promises of “Big Data” in education is to use non-experimental data to discover insights. We focus on studying the impact of example usage in a Java programming tutoring system using observational data. For this, we compare different formulations of a recently proposed generalized Knowledge Tracing framework called FAST. We discover that different formulations can have the same predictive performance; yet their coefficients may have opposite signs, which may lead researchers to contradictory conclusions. We discuss implications of using fully data-driven approaches to study non-experimental data

An efficient RGB-UAV-based platform for field almond tree phenotyping: 3-D architecture and flowering traits

[Background] Almond is an emerging crop due to the health benefits of almond consumption including nutritional, anti-inflammatory, and hypocholesterolaemia properties. Traditional almond producers were concentrated in California, Australia, and Mediterranean countries. However, almond is currently present in more than 50 countries due to breeding programs have modernized almond orchards by developing new varieties with improved traits related to late flowering (to reduce the risk of damage caused by late frosts) and tree architecture. Almond tree architecture and flowering are acquired and evaluated through intensive field labour for breeders. Flowering detection has traditionally been a very challenging objective. To our knowledge, there is no published information about monitoring of the tree flowering dynamics of a crop at the field scale by using color information from photogrammetric 3D point clouds and OBIA. As an alternative, a procedure based on the generation of colored photogrammetric point clouds using a low cost (RGB) camera on-board an unmanned aerial vehicle (UAV), and an semi-automatic object based image analysis (OBIA) algorithm was created for monitoring the flower density and flowering period of every almond tree in the framework of two almond phenotypic trials with different planting dates.[Results] Our method was useful for detecting the phenotypic variability of every almond variety by mapping and quantifying every tree height and volume as well as the flowering dynamics and flower density. There was a high level of agreement among the tree height, flower density, and blooming calendar derived from our procedure on both fields with the ones created from on-ground measured data. Some of the almond varieties showed a significant linear fit between its crown volume and their yield.[Conclusions] Our findings could help breeders and researchers to reduce the gap between phenomics and genomics by generating accurate almond tree information in an efficient, non-destructive, and inexpensive way. The method described is also useful for data mining to select the most promising accessions, making it possible to assess specific multi-criteria ranking varieties, which are one of the main tools for breeders.This research was funded by the AGL2017‑83325‑C4‑4R (Spanish Ministry of Science, Innovation and Universities, EU‑FEDER Funds), Transforma (ref.: PP.TRA.TRA.2016.00.6; PP.TRA.TRA.2019.00.2), UAVarAL (Ref. 1264556, Programa Operativo FEDER 2014‑2020 and Consejería de Economía y Conocimiento de la Junta de Andalucía) and INTRAMURAL 201840E002 (CSIC funds) projects.We acknowledge support of the publication fee by the CSIC Open Access Support Initiative through its Unit of Information Resources for Research (URICI)Peer reviewe

Dynamics of Ecosystem Services during Forest Transitions in Costa Rica

International audienceThe forest transition framework describes the temporal changes of forest areas with economic development. A first phase of forest contraction is followed by a second phase of expansion once a turning point is reached. This framework does not differentiate forest types or ecosystem services, and describes forests regardless of their contribution to human well-being. For several decades, deforestation in many tropical regions has degraded ecosystem services, such as watershed regulation, while increasing provisioning services from agriculture, for example, food. Forest transitions and expansion have been observed in some countries, but their consequences for ecosystem services are often unclear. We analyzed the implications of forest cover change on ecosystem services in Costa Rica, where a forest transition has been suggested. A review of literature and secondary data on forest and ecosystem services in Costa Rica indicated that forest transition might have led to an ecosystem services transition. We modeled and mapped the changes of selected ecosystem services in the upper part of the Reventazón watershed and analyzed how supply changed over time in order to identify possible transitions in ecosystem services. The modeled changes of ecosystem services is similar to the second phase of a forest transition but no turning point was identified, probably because of the limited temporal scope of the analysis. Trends of provisioning and regulating services and their tradeoffs were opposite in different spatial subunits of our study area, which highlights the importance of scale in the analysis of ecosystem services and forest transitions. The ecosystem services transition framework proposed in this study is useful for analyzing the temporal changes of ecosystem services and linking socio-economic drivers to ecosystem services demand at different scales